(S//SI//REL) Is There a Sustainable Ops Tempo in S2? How Can Analysts Deal 



with the Flood of Collection? — An Interview with 



(conclusion) 



FROM: the SIDtoday Editor 
Run Date: 04/06/2011 



(U// FQUQ) Editor's n ote: Here's the conclusion of our interview with SID's Deputy Director for 
A&P^^^^^^^^f(pictured). (If you missed part one , you can find it HERE .) In this segment 
Hf^^^^Eomments on Analytic Modernization and why it is taking center stage... 



6 . Q: (U) When we last interviewed you in August 2009 — you were Assistant DDAP 
at the time —you spoke of the importance of creating a sustainable ops tempo in S2 . 

Have you succeeded? 

A: (U). No. No, I haven't... And it’s not just because there’s so much happening in the world. Here's 
the thing: we have to take a fundamentally different approach to how we do business. The volume 
and quality of access we have now gives us an unprecedented capability to produce some of the 
best intelligence we ever have, but the fact is that we are only getting to a fractional portion of what 
we have access to. I can't tell you for sur e that as good as what we produce is, that it's the absolute 
best we can do. It's not sustainable for die workforce or the mission to keep working in the way we 
always have. 

(U//FOUQ) We've embarked on Analytic Modernization. In the past there have been 
transformations... modernizations... renovations... it gets tiring! But to a large degree those efforts 
have represented supercharging existing capabilities — doing more with less. We need to 
fundamentally change how we interact with SIGINT. 

(U//FOUO) This is a big deal, because for decades we've trained our workforce — and took pride in 
the fact — that we had the unique responsibility within the IC to manage this information in 
accordance with the 4th Amendment. To that extent we thought of SIGINT as "radioactive 11 and it 
was our job to render the information safe for others. Our process was about managing the 
information in a manner consistent with the privacy rights of U.S. persons. 

(U//FOUO) I believe we have developed an awareness of the information and an understanding of 
our authorities diat allows us to think differently about that relationship... and in so doing, to create 
an increased capacity and recover more time for the analysts. I don't mean only analysts in SID, but 
also analysts in IAD , in NTOC, etc. The key is to better leverage the Intelligence Community and 
our partner relationships for the exploitation of SIGINT, which is something in the past we would 
not have done to the degree we're proposing -- it was considered our domain (and "radioactive"). 

(U//FOUO) We must take advantage of the expertise and capabilities in die IC and our customer 
base to enhance discovery and capacity, and to make the actionable information available almost as 
soon as we encounter it. This is not about turning NSA into a collection resource for others 
(although collection is in fact one of our great strengths and one we are uniquely qualified to 
undertake) — it's about making sure we don't expend cryptologic resources doing work that is not 
uniquely cryptologic in nature — work that others can do. 




(U) Q: Can you $ve an example? 



A: (S//SI//REL) If NSA discovers a pathway into an adversary's information space and we extract a 
terabyte of CAD [computer-aided design] drawings of weapon designs, is it of best value for the IC 
to have talented NSA analysts work their way through this 1 terabyte of data? Is it uniquely 
cryptologic? Or is it of better benefit to the nation that we expose it immediately to the best and 
brightest weapon designers in the US government to work on in a collaborative space, to triage, 
assess and exploit the value of that information? 

(U//FOUO) In so doing we are not precluding NSA analysts from continued access to the data, any 
more than a published SIGINT product report isn't available for future reference. We are just 
leveraging the power and expertise of others so that we can turn our attention to that that which 
only we can exploit by virtue of our unique talents. 

Q: (U) Isn’t it possible that our customers won’t want to devote their own people 
and time to analyzing die data? 

A: (U//FOUO) If we get access to the best information on a topic, the value will speak for itself and 
the customers will make the resources available. It's like YouTube videos - they "go viral" when 
people with good reputations recommend a video to others. If we put the information out there and 
monitor the customer response to it, we'll know when to recommend specific items they might 
want to take a close look at. Amazon does the same thing by looking at how you react to the 
products you've looked at. We can figure out what the communities of interest are for that topic... If 
the material is in a foreign language, we have tools they can use to get the gist of it, and if it looks 
promising, they can use the National V i rtual Trans lat ion Center to get it translated. We want our 
language analysts focused on uniquely cryptologic problems that rarely boil down to a 
straightforward translation. 

(U//FOUO) So what does this approach accomplish? We've exposed the intelligence to people who 
can interpret it and use it, and we've created opportunities for collaboration. We've also off-loaded 
the responsibility to manage the data in our respositories and own the compliance responsibility. If 
data is stored in the Library of National Intelligence , someone else is paying for it. 

(U//FOUO) In the end, we exist to produce information. The only way to go more deeply into 
targets is to avoid getting stuck on production that others can do for themselves. Our challenge is to 
be always out looking for something new. We need to think about problems, not just about 
production. 

(U//FOUO) The collaborative component means it's not an NSA view, a DIA view, etc... Rather, it's 
opened up to all on A- Space. I would like to see people log onto A-Space and announce "I want to 
share traffic and create a multi-seal report based on joint input." Why not collaborate at the point of 
discovery? 

Q: 7, (U//FOUO) Various pushes for analytic modernization have been going on 
for decades at NSA, but now the issue really seems to be taking center stage. In 
fact* the number one r ‘ SIGINT Goal for 2011-2015 " is to Tl revolutionize analysis. Tl 
What’s different now? 

A: (S//SI//REL) We live in an Information Age when we have massive reserves of information and 
don’t have the capability to exploit it. I was told that there are 2 petabytes of data in the SIGINT 
System at any given time. How much is that? That's equal to 20 million 4-drawer filing cabinets. 
How many cabinets per analyst is that?? By the end of this year, we'll have 1 terabyte of data per 



second coming in. You can't crank that through the existing processes and be effective. 

Q: (U) ,„So it’s a matter of volume? 

A: (S//SI//REL) Not volume alone, but also complexity. We need to piece together the data. It's 
impossible to do that using traditional methods. Strong selectors — like phone numbers — will 
become a thing of the past. It used to be that if you had a target's number, you could follow it for 
most of your career. Not anymore. My daughter doesn't even make phone calls, and many targets 
do the same. Also, the commercial market demands privacy, and this will drive our targets to go 
encrypted, maybe into unexploit able realms. Our nation needs us to look for patterns surrounding a 
particular spot on Earth and make the connections — who can do that if not us? And we can't do it 
using traditional methods. 

8. Q: (U) Looking into the future, is there any tiling that especially worries 
you? „,An eventuality (internal or external) that would make it hard for A&P to 
continue to put out quality intelligence? 

A: (U//FOUO) I'm worried that we have so much good stuff that we could lock down analysts and 
have them just producing product, and something would jump out and surprise us. So we need the 
discipline to invest in the wild and the unknowns. 

9. Q: (U) Is tli ere any tiling else you'd like to say to the SIGINT workforce? 

A : (U) At the risk of being melodramatic, as hard as the work is and as challenging as the work 
environment can be, we should never forget how lucky we are to have a job where we can have so 
much impact for the nation. Don't let the turkeys get you down! 



(U) The SIGINT Philosopher: Cognitive Overflow? 



FROM: (U//FOUO 
Run Date: 04/15/2011 

(U) There’s a computer sitting atop your shoulders. Granted, real computers can apparently best 
human brains on Jeopardy with ease, but all the same... Since Noam Chomsky and his cohorts at 
MIT opened the floodgates to the study of how we think, cognitive psychology has come a long 
way. Although the ensuing decades of research have highlighted the astounding capabilities of our 
gray matter, the field has also exposed the limitations our brains are subject to. It may be worth 
considering the implications these limitations have for our work in SID. 

(U) "Channel capacity" is the term some cognitive psychologists have begun to apply to the brain’s 
limits on the amount of certain information it can retain. For instance, research shows that the 
average person can only differentiate between 5-9 different tones, shapes, or textures at a given 
time. Any more, and our capacity to categorize becomes overtaxed, and we begin to make 
mistakes. In other words, the servers overload. It is yet another example of how unprepared 
humans are, in evolutionary terms, for the information age. Evolutionary biologist Sherwood 
Washburn once wrote: 

(U) "Most of human evolution took place before the advent of agriculture, when we lived in small 
groups , face-to-face. Man evolved to feel strongly about few people , short distances, and relatively 
brief intervals / 1 

(U) The question then becomes: If an individual brain has finite "channel capacity," does the vast 
collective of SID, comprised of thousands of brilliant, yet limited, brains also have a definite 
"channel capacity"? If so, what is it? How do we know when we’ve reached it? What if we've 
already exceeded it? In essence, could SID’s reach exceed its grasp? Can the combined cognitive 
power of SID connect all the necessary dots to avoid, predict, or advise when the improbable, 
complex, or unthinkable happens? 

(U) Take for example the number of tools, clearances, systems, compliances, and administrative 
requirements we encounter before we even begirt to engage in the work of the mission itself . The 
mission then involves an ever- expanding set of complex issues, targets, accesses, and capabilities. 
The "cognitive burden," so to speak, must at times feel overwhelming to some of us. The SID is an 
organism with many moving parts. So how do we ensure our SIGINT potential is in line with, and 
doesn't overwhelm our collective cognitive capacity ? Can we count on our overarching SID 
mechanism to self -regulate, to organically cull, sort, and retain? Or is there perhaps something 
extra we ought to be doing to ensure we operate at full exploitative and analytic force? 

(U) Surely someone will point out that the burgeoning amalgam of technological advances will aid 
us in shouldering the burden. However, historically, this scenario doesn't seem to completely bear 
out. The onslaught of more computing power— often intended to automate some processes— has in 
many respects demanded an expansion of our combined "channel capacity," rather than curbing the 
flow of the information that’s necessary to retain. 

(U) It’s an issue worth thinking about and discussing. In the meantime, I’ll be working on my 14- 

character p assword. .. 



(U) Editor’s note: See a Tapioca Pebble on this topic. 




(U) ’Data Is Not Intelligence’ 



SID Reporting Board (S12R) 
Run Date: 09/18/2007 



(U//FOUO) These words came from Dr. Thomas Fingar (pictured) in his keynote address at the 
Analytic Transformation Symposium in Chicago on 5 September. Such a strong reminder at the 
opening of his address was intended to remind those at the symposium of the importance he and the 
Director of National Intelligence, the Honorable J. Michael McConnell, place on improving 
analysis throughout the Intelligence Community. 

(U//FOUO) Dr. Fingar, the Deputy Director of National Intelligence for Analysis, made this 
statement at the opening of the symposium sponsored by the Intelligence and National Security 
Alliance, a non-profit, non-partisan public policy forum focusing on intelligence and national 
security issues. The symposium was held in Chicago, Illinois, from 4 to 6 September 2007. 

(U//FOUO) Dr. Fingar continued by saying that "intelligence comes from the brains of analysts." 

H e clearly wanted those attending the symposium to understand his view of the importance of the 
analytic process in producing intelligence. The emphasis throughout his remarks was that the 
Intelligence Community must transform its analytic mission. The transformation is being effected 
in three areas: enhancing the quality of analytic products; managing the mission more effectively at 
a Community level; and building more integrated analytic operations across the Intelligence 
Community. 

(U//FOUO) To enhance die quality of analytic products, analysts themselves must improve. They 
can do this by receiving more and better formal training, and by continuing to learn through 
experience and mentoring from more experienced analysts. In addition, diey must alter mindsets 
that keep them from sharing information, especially that which would improve an intelligence 
product. An adjunct to changing mindsets about sharing information is establishing trust between 
and among analysts as a way to improve the quality of analytic products. 

(U//FOUO) In an explanation of how to manage the analytic mission more effectively at the 
Community level, Dr. Fingar reviewed the A-Space and Library of National Intelligence (LNI) 
programs. While some leaders might consider these two programs more as tools, Dr. Fingar 
stressed that they were programs to help analysts enhance products. A-Space will provide a virtual 
environment in which analysts can work on data and collaborate. The LNI will give analysts a 
research facility diat will help them gather already-disseminated intelligence on a topic. 

(U//FOUO) The effort to build more integrated analytic operations involves, in part, gready 
improving collaboration. Setting common standards is a key to collaboration, and collaboration 
will enhance the quality of analytic products, according to Dr. Fingar. H e emphasized that the IC 
analytic standards recently approved were a step, but only a step. He called for "transparency" in 
intelligence analysis; that is, that all analysis has to be reproducible. Following established 
common standards will help ensure transparency. More importantly, collaboration will help 
establish an analytic community. 

(U//FOUO) Dr. Fingar's address set die tone for the rest of the symposium. The point was that the 
quality of intelligence products must improve— must "transform." The most important part in the 
transformation is the analyst. In training analysts better, by encouraging them to learn continually 
through experience and mentoring, product will improve. More effective management, through 




programs such as A-Space and the LNI, will help give analysts data and intelligence they need, and 
a better environment in which to work. Collaboration is encouraged and made easier by these 
programs, and collaboration is part of building integrated operations. All of these together will help 
ensure that the quality of analytic products improves— that customers receive intelligence, not data. 




(U) T ’Signal v Noise” Column: Do We Need a Bigger SIGINT Truck? 



FROM: (U//FOUO)J| 

Run Date: 01/23/2012 

(U) Once upon a time, two people formed a new business. "They built a small shed beside a busy 
road, rented a truck and drove it to a farmer's field, where they purchased a truckload of melons for 
a dollar per melon . They drove die loaded truck to their shed, where they sold them for a dollar per 
melon ... As they drove back toward the farmer's field to get another load, one partner said to the 
other, "We’re not making much money on this business, are we?" The partner replied, "Do you think 
we need a bigger truck?"* Sometimes more is just more, if you're not paying attention to what the 
metrics are telling you, 

(U) This example is a pretty obvious demonstration of the need for business intelligence. The point 
of business intelligence is diat "what gets measured, gets managed" If we don't have the right 
measurements available, we simply guess . In this example, the melon duo felt like diey should just 
sell more melons to increase profits and to do so meant getting a bigger truck (don't confuse them 
with the facts), 

(S//SF/REL) Throughout SID we often discuss what "a pound of SIGINT" costs or is worth. It's 
difficult, but not impossible, to figure out the value or cost of a specific pound of SIGINT. 
"Measuring lets us determine exacdy what we can and can't control. Then what we can control, we 
fix; what we can't control, we either change or isolate. By measuring, we know where to spend our 
time and energy."** Figuring out what to count across the Enterprise and how to obtain those 
count results from our SIGINT world can take hours, multiple tools, a variety of frightening 
pivot tables, multiple queries, a small army of ”phone-a- friends , " and possibly a registry action , 
(I am not kidding.) Ironically, for an intelligence organization, operational business intelligence can 
be hard to come by. 

(S//SF/REL) Recently I tried to answer what seemed like a relatively straightforward question 
about which telephony metadata collection capabilities are the most important in case we need to 
shut something off when the metadata coffers get full. By the end of the day, I felt like capitulating 
with the white flag of, "We need COLOSSAL data storage so we don't have to worry about it" (aka 
we need a bigger SIGINT truck), because getting the metrics for empirical evidence to review was 
so very difficult and, frankly, I'm still a little scarred by the experience. 

(S//SI//REL) Without metrics, how do we know that we have improved something or made it 
worse? There's a running joke in the S3 community that we'll only know if collection is important 
by shutting it off and seeing if someone screams. That's one metric anyway, and sadly, die easiest 
one (and arguably the most effective) to obtain. 

(S//SF/REL) So, in this year of decreasing budgets and ever-constrained resources, it's naturally 
more important to measure what we can to optimize our time and energy. Wherever you sit and 
whatever grade you clock in at, there are things you can measure to determine how effectively you 
are doing what you are doing. Start measuring today and set some new 2012 goals to improve your 
contributions. For a head start, I suggest reading " How to Measure Anything: Finding the Value of 
Intangibles " by Douglas Hubbard. As an added benefit, this skill will not only help improve our 
SIGINT business but also dramatically improve your EPA, resume, and ACE bullets! After all, it's 
one thing to say you need a bigger SIGINT truck, new tool, promotion, etc. and another thing 
altogedier to prove it. 




(U//FOUO) Have thoughts oil this topic? Post them on the related Tapioca Pebble . 

(U) Notes: 

* (U) "Do you think we need a bigger truck?" (Source: 

http://wwv.hort.wisc.edu/cran/pubs_archive/newsletters/2002/news _2002_09_05.pdf), 

** (U) Bethune, G. (1990), " From Worst to First: Behind the Scenes of Continentars Remarkable 
Comeback ," 

(U//FOUO) Standard disclaimer: All SIDfodqy columns express the views of the columnist alone 
and do not represent the official views of any NSA organization. 



(U) Op-ed: Leave Bright Pebbles, Not Breadcrumbs, for Those Coming After 
You 



FROM: ( U//F O U O) 

Chinese Voice/Graphic Language Analyst 



Run Date: 09/24/2010 



(U//FOUO) Editor's note: In this op-ed piece, 
to document and share what we know. 




provides some thoughts on the need 



(U) "".pure amount of signal-to-noise that people have to sift. " 

(U//FOU0) This quote appeared on Tapioca recently. It caught my attention because we are 
drowning in information. And yet we know nothing. For sure. 

(U) Anyone know just how many tools are available at the Agency, alone? Would you know where 
to go to find out? 

(U) Anyone ever start a new target or a new language or IA [Intelligence Analysis] tour/internship 
without the first clue where to begin? Did you ever start a project wondering if you were the sole 
person in the Intelligence Community to work this project? How would you find out? 

(U//FOUO) How about vocabulary or target knowledge? What if your management feels that 
linguists are fungible assets to be moved around where necessary, target expertise and 
breadth/depth of knowledge be damned? 

(U//FOUO) Let us assume that you might know a few places to look: MAUI, Wordscape . or a 
relevant classified wiki page on the Agency's networks or even — gasp — the Tar get Knowledge 
Base (TKB). 

(U//F0UO) Here is where the nightmare begins: What if the information you so desperately seek is 
not there? Do you keep turning to your podmates (who might also be new to the job) or to your 
mentor or to the "target guru 11 in your Product Line (who might not be willing to help you after the 
20th time)? 

(U//FOUO) Let us continue with the nightmare: The information is not available because no one 
put it there. No wiki, no Wordscape entry, nothing in TKB. 

Why? 

(U//F0U0) Here are some "reasons": 

1. "Our branch chief looked at it [TKB] and determined it was not user-friendly. So well 
continue to use our Excel spreadsheets." 

2. "I have way too much work to do to feed TKB all day. I might send around mass emails, 
however." 

3. "I keep my vocab on a flat file." 



4. "We're not allowed to go to or post anything on A-Space ." 



(S//SI//REL) Result: You're out of luck: 

1. The phone number or name you're looking for is not in TKB. Never mind the lack of any 
attachments or extra information even if the phone number does happen to return as 
"entered." 

2. The vocabulary item (English or target language) is not in Wordscape. And you know 
sweet-fanny- all about nuclear power. 

3. You never find out that, on A-Space, there are groups of target experts champing at the bit 
to share their expertise. And who have posted reports, answered questions, set up blogs, and 
even "tagged" key items. 

(U//F0UO) It is imperative we begin sharing — permanently — what we know. The days of 
showing up at NS A, opening your desk drawer, and then locking up that very drawer 30 years later 
-- are gone. The days of short tours, internships, "fungible asset" deployments, are the new rule. 
Email, so new to some of us over 20 years ago - is now viewed as quaint. 

(U//FOUO) What can you do, right now, on your own, to leave bright pebbles rather than 
breadcrumbs for those who follow? 

Simple: 

■ Start populating a database near you: TKB, Wordscape. 

■ Don't know anything about TKB? " Go Firefly " and the TKB people are willing to 
come right to your desk. 

* Wordscape: type "wi Wordscape" [takes you to the wiki page for Wordscape l. 

■ There are links to various language dictionaries you can start populating! Today! 

■ Start a JournalNSAblog : Share your expertise. Engage with other bloggers. 

* Create a wiki page for your area of expertise. 

* Visit " Connexions " for a way to dump your bookmarked sites and tag them so others can 
just click to find the information that has been helpful to you. 

* Start visiting A-Space . Join some groups. Answer questions your IC partners posts. 

(U) In short: Don't let those coming behind you suffer the way you have. 



(U//FOUO) Have thoughts on this topic? Post them on the SID today Blog . Comments on the 
article itself can be submitted using the "Comments/suggestions about this article" button below. 



(U) The SIGINT Philosopher: The Fallacies Behind the Scenes 



FROM: (U//FOUO)H^^^H| 

Run Date: 03/15/2011 

(U) At the end of the day, much of the work done in SID revolves around getting information to 
people who need to make a decision. Often that decision involves making an estimate as to how 
probable or plausible a given event is, and what implications such a hypothetical event might have 
on our national security. It seems straightforward enough. However, at the outset, there are 
undercurrents flowing through the minds of every analyst, before the fingers even hit the keyboard. 
The analyst might not even know they're at work, but they are. They're cognitive fallacies. 

(U) There's no blame to be handed out — these fallacies are hard-wired into our brains as a result of 
thousands of years of evolutionary instinct and decision-making to simply survive. In relative 
historic terms, we've only very recently shifted our attention from such perfunctory tasks to more 
complex avenues. So, in the analysis we undertake in our daily duties, here are a few biases to be 
aware of, or perhaps attempt to change, on an individual and collective level: 

The Texas Sharpshooter Fallacy 

(U) This logical fallacy is named for a Texan who shot a hole in his barn, drew red and white 
circles around it, and gathered the neighbors to brag about how great a shot he was. While finding 
significance and connections between people and events is what intelligence is all about — the 
downside is that sometimes we stretch to make the case that relationships or significance are 
present where they aren't. When we make the case first, the evidence we gather tends only to 
reinforce it. The reverse should be the rule: go where the evidence leads, don't lead the evidence. 

The Gambler's Fallacy 

(U) Kenny Rogers knew when to hold 'em. So should we. Since we're hard -wired to look for 
patterns, we have a tendency to think of numbers and events as being orderly, when they're 
anything but. Neuroscientists call this phenomenon apophenia. For instance, if you're taking a turn 
at the roulette wheel, how many hits on black will it take before you think to yourself, "We're due 
for a red..."? Or perhaps you think because a number on the wheel has come up so often in die past, 
it's bound to be more absent in die future? Pioneering cognitive scientist Amos Tversky noted "It's 
natural to he inclined toward patterns when we observe cause and effect. But in nature , events 
rarely happen with equal frequency Events are rarely ever "due" to occur, and the frequency of an 
event in the past is not always an indicator of its future appearance. 

The Sunk-Cost Fallacy 

(U) How many times have you been watching a terrible movie, only to convince yourself to stick it 
out to the end and find out what happens, since you've already invested too much time or money to 
simply walk away? This "gone too far to stop now" mentality is our built-in mechanism to help us 
allocate and ration resources. H owever, it can work to our detriment in prioritizing and deciding 
which projects or efforts are worth further expenditure of resources, regardless of how much has 
already been "sunk." As has been said before, insanity is doing the same thing over and over and 
expecting different results. 



The Appeal to Probability 




(U) Remember Y2K? We fetched our gas masks and stocked up on canned food for the impending 
doom that would come at the dawn of the millennium. But alas, midnight came and went with nary 
a global meltdown. That needless commotion might have been attributed to an appello 
probabilitatem, where an event is thought to be possible , and is hence erroneously determined to be 
plausible. In other words, it's the false assumption that because something can happen, it likely will 
happen. While preparation is a noble goal -- bear in mind the actual probabilities. 

(U) There are hundreds of other fallacies that I could touch on, but the main point is to engender 
awareness that our decision-making processes are directly and unconsciously affected by nature's 
own hard- wiring. Keeping that fact in mind could have positive impacts on the way we choose to 
execute our mission. Who knows, perhaps there are cognitive fallacies that have developed - not 
resulting from many generations of human evolution — but from decades of SIGINT evolution? It 
may be worthwhile to consider the unintended effects on our collective processes that such an 
evolution may have had, and be on guard against cognitive fallacies and logical biases of all 
stripes, whether they be recent, or as old as humanity itself. 

Now -- back to my poker game with Kenny... 



(U) Have thoughts on this topic? See a related Tapioca Pebble . 



(U) SIGINT Mission Thread 3 



FROM: ( U./7F O UO ) 

Thread Three Lead (S2) 

Run Date: 08/01/2011 

Achieving a Balance Between Discovery and Production 

(U//FOUG) The key to good decision making is not knowledge. It is understanding. We are 
swimming in the former. We are desperately lacking in the latter. 1 ’* In the afterward to his 2005 #1 
national bestseller Blink — The Power of Thinking Without Thinking , author Malcolm Gladwell 
provides his perspective on die danger of confusing information (collection) with understanding 
(analysis). Gladwell has captured one of the biggest challenges facing SID today. Our costs 
associated with this information overload are not only financial, such as die need to build data 
warehouses large enough to store die mountain of data that arrives at our doorstep each day, but 
also include the more intangible costs of too much data to review, process, translate, and report. 
SID's first strategic goal for 2011-2015, the challenge to revolution analysis, is aimed squarely at 
this tension between information and understanding. 

(U//FOUO) In order to revolutionize intelligence, we must "fundamentally shift our analytic 
approach from a production to a discovery bias, radically increasing operational impact across all 
mission domains."** With so much data at our fingertips, we must learn how to push the lesser 
value data to the side, move data that needs less analysis directly to our customers, and provide 
ourselves the needed agility to dig deep into the toughest analytic problems to produce 
understanding from well-hidden information. 

(U//FOUO) SIGINT Mission Thread 3 challenges us to "achieve a balance between discovery and 
production that enables analysts to maximize their focus on uniquely cryptologic problems, to 
include large-scale data analysis and the discovery of new targets and their communications."*** 
Our first task in achieving this goal is to build a strong foundation of existing analytic techniques. 
The Analytic Body of Knowledge (ABoK, for short) defines common terminology and captures 
existing analytic methodologies. The ABoK web portal opened for business on 17 May 2011, and 
has seen a steady increase in use as your fellow analysts document useful tradecraft and explore 
new ideas. Through 13 June 2011, over 100 analysts have visited the ABoK portal. You can learn 
more about the ABoK by visiting the web portal directly or reading the accompanying Wikilnfo 
article . 

(U//FOUG) In a closely related activity, we are also working to document the depth and breadth of 
analytics and the function they perform or will perform. This knowledge is being captured in the 
Analytics Taxonomy (A-Tax, for short). Once substantially populated, A-Tax will provide the 
mechanism to effectively track and promote collaborative analytic development across the 5-eyes 
enterprise. The number of users accessing the taxonomy has more than doubled to over 80 just in 
the last three weeks. Working together, the ABoK and A-Tax will provide analysts access to and 
influence over shared analytic tradecraft and automated analytics. We invite you to visit the ABoK 
and A-Tax portals and start documenting existing tradecraft and analytics. 

(U//FOUG) We will use the SigDev Training Roadmap to baseline existing discovery skills across 
the analysis enterprise, identify critical knowledge gaps, and work to provide the necessary training 
and skills needed to develop new tradecraft. These advanced tradecrafts could include things such 
as target behavioral modeling and cyber attribution. The training will also include familiarization 
with new discovery frameworks such as those described in the Question Focused Dataset (QFD) 




Strategy and the Revised Metadata Strategy . These strategies define on-going additions to the 
corporate analytic architecture. Of course, NS A and the FVEY community have been coming up 
with (and implementing) QFDs for years. What has long been missing is a central location where 
QFDs and related analytics can be easily shared, discovered and evaluated among the analytic 
community. That's the basic idea behind GAMBIT, a new prototype web portal that will serve as a 
key enabler of the AMQD QFD Strateg y. GAMBIT is slated for an alpha release in late July. We 
encourage you to learn more about it on the AMQD QFD Strategy SpySpace page . 

(S//SF/REL) In addition, over the next 18-24 months, we will work with each of the S2 GCMs to 
develop product line discovery strategies, to help them shift emphasis towards discovery activities 
and to increase the customer impact of our intelligence production. 

(S//SF/REL) SIGINT Mission Thread 3 is working to empower the enterprise to leverage our 
existing tradecraft arsenal, to equip the enterprise with the capability and skills to forge new 
tradecrafts, and to imbue the enterprise with a new culture that values and encourages the 
development of understanding over the collection of information. Please join us in the journey. 



(U) Footnotes 
*p. 256. 

** SIGINT Strategy: 2011-2015 (1/25/2011), http :/y^^^^^^^^^ J/fi!es/l 027/p df/2 0 1 1- 
01-25 SIGINT Strategy.pdf Hast visited 6/13/2011), p. 4. 



*** AMOD_FYll.12_Deliverables.Febll.pptx. 



(U) The SIGINT Philosopher: In Praise of Not Knowing 



FROM: (U//FOUO 
Run Date: 08/1 G/2 012 

(U) I have to begin with a complaint. Hopefully, this will not incite a war with our IAD comrades, 
but my beef is really with IT types. You've ruined a great word. If you Google the word "agnostic 11 
on your NSANet browser right now, you will find the results rife with tecli-y terms like these: 

■ The PW storage environment is agnostic regarding... 

■ Comcast's protocol- agnostic,,, 

* Application layer is bearer agnostic... 

(U) ...and so on. In all those uses, "agnostic 11 means something like "indifferent to." This tech-y use 
of die word has even snuck out into die common vernacular; a leader once called on SID to become 
"geographically agnostic." 

(U) But that's not what the word means. As the classically trained analyst who sits across from me 
will attest, its Greek roots literally mean "without knowledge." An "agnostic" in religious terms is 
someone who is uncertain about God's existence, and who possibly believes die question is 
unknowable. But that doesn't mean agnostics don't care whether God exists. 

(S//SI//REL) I ascend diis arcane etymological soapbox not just because a word has been abused. 
Words change meaning in any language, and there's naught I can do about it. But the admission 
that one does not know something is really an excellent intellectual exercise . I recall one time, 
while stationed at SUSLAK . when I briefed the South Korean Ministry of National Defense on a 
North Korean military exercise. The South Koreans had just gotten done giving eleven hypodieses. 

I explained simply that we weren't sure. The General looked at his countrymen and said, "If you 
don't know, say you don't know like the Americans." 

(C//REL) The IC is full of questions to which we do not know the answer. And yet we are smart, 
ambitious, conscientious people who were hired to answer those questions, dagnabit, and so we go 
about trying to answer them industriously. We can take one of two approaches. First, we can write 
our best guesses, and couch them with all sorts of qualifying language. This is the approach some 
take with North Korea. Although we may know less about North Korea than any country on earth, 
hardly a day goes by when my "highlights of IC reporting" e-mail doesn't have a North Korea 
report in it, in which some industrious thinker has just pontificated that Kim Jong-Un might be 
open to the West because Mickey Mouse appeared in a North Korean parade. Just by volume, you'd 
think we actually knew a lot about this country. 

(U//FOUO) NSA often takes a second approach, where we attempt to address a very small subset 
of an important question. Say flier e is an intelligence need (IN) for Zendian economic information. 
People want to know how the main sectors of the Zendian economy are performing, whether the 
needs of the people are being met, and what the high-level plans are to direct the economy. 
Reasonable questions. But NSA doesn't have the access to answer them. Their reclusive leaders do 
not use electronic communications we can target. Their missions abroad use triple pig-Latin 
encryption, and underlying it all is the darn Zendian language that nobody here can even speak. So 
what do we do? Well, it turns out we can get access to a Zendian gum-chewing manufacturer that 
has its operations based in the North Pole. So our Zendian Econ team writes 132 reports on 
Zendian gum sales under the Zendian economic IN. Analysts in other agencies are so happy to get 




anything, they report that they are grateful for the reporting, which leads leadership here to 
conclude the team is doing its mission. 

(U//FOUO) Now, I've been in positions before where I was working the equivalent of the Zendian 
gum factory. There wasn't much I saw of use in the collection I had to work with. And if that's you, 
and that's where you find yourself, you absolutely should try to do something rather than nothing. 
You're a civil servant, and you should do the best you can to give value for your paycheck. 

(U//FOUO) But it would also be nice for those laboring on these kinds of missions to have their 
leader s occasionally recognize how little we really know.* Often, in their understandable desire to 
keep morale high and fight for resources, they tend to always focus on die positive; "We got 87 
positive feedback reports from customers! Our customers love our stuff!" That's fine, but going 
overboard with this sort of thing can lead analysts to conclude they have gone crazy. Occasionally 
hearing "you know, we just aren't able to answer the questions our customers really want” can 
actually be a refreshing bit of honesty. A little (real) agnosticism would really help. 

(U) This kind of honesty can help to "rip off die bandage" and reveal how difficult the task of good 
intelligence gathering really is. It can prevent throwing good money after bad when leadership 
believes its current efforts are working. It also helps maintain our ethos as honest, straight-shooting 
Americans. 

(U) But, you know, whatever. I'm agnostic to die whole thing. 



(U) Note: 

*(U//FOUO) I realize some of you have great access and really can answer a lot of your customers' 
questions. For you, just know that I hate you, and click on over to a recent Si gnal v Noise . 



(U) ELINT Analysts - Overcome by Overload? Help is Here with IM&S 



FROM: (U//FOUO)||^^H 
ELINT Modernization IM&S (BAG) 
Run Date: 10/03/2011 



(S//REL) Globalization is leading to an unprecedented proliferation of complex weapon systems. 
This, coupled with increasingly sophisticated SIGINT collection systems, is leading to exponential 
growth in the volume, velocity, and variety of ELINT collection and processing artifacts. ELINT 
Modernization (EMOD) Information Management and Storage (IM&S) provides long-term, 
uniform access to an increasing number of ELINT data sources, to include traditional (i.e., satellites 
and surveillance aircraft) and non-traditional (i.e., UAVs, fighter jets, naval vessels) collection and 
processing systems. 

(U) Currently, an IOC* system is deployed which will provide access to five years of ELINT 
reports and raw data to analysts with NSANet access. Thin client applications allow analysts 
around the world to search for and retrieve ELINT data with only an NSANet terminal and a 
current version of Mozilla Firefox. Many additional features are designed to invert the paradigm 
where an analyst spends more time searching for data than analyzing it. These include: 

* (U) Thumbnail tools which provide users the capability to preview Power Spectral Density 
(PSD) displays of Digital Intermediate Frequency (DIF) files with links that enable file 
slicing prior to downloading them for further analysis. 

* (U) Subscriptions to automate the query process enabling data to be staged and/or 
delivered for use when an analyst starts his/her work for the day. 

■ (U) PL3+ Accreditation to enable data sharing with our mission partners 

■ (U) User uploads to enable data sharing within the community. 

* (U) Linking to present relations between data. 

* (U) Translators to convert data from native format to standard formats used by the analytic 
tools. 



(S//REL) By early 2012, additional archives will provide access to Combat Sent, Specific Emitter 
Identification (SEI), Over-the-Horizon-Radar (OTHR) and other conventional sources. Web 
technologies are used to present a single consolidated view of the data without regard to its actual 
location. The system will also connect other reference data sources such as Combined Emitter 
Database (CED) allowing users to view reference data in support of their analysis. 



(U) To request an IM&S account from an NSANet terminal, click 1ERE . For computer-based 
training, click HERE and then search for TOOL2065. 

(U//FOUO) POC: ^^^HpM IM&S, secure: 

*(U) Initial Operational Capability 






(U) "Signal v. Noise" Column: Summit Fever 



FROM: (U//FOUO)|^^^^^^^^^| 

Run Date: 06/25/2012 

(U) Imagine that you have decided to climb Mount Everest. You train for a few years, buy about 
$8,000 dollars worth of equipment and $3,000 of bottled oxygen, not to mention the training time, 
and then finally are ready to go. You buy the airline ticket for Nepal, maybe even go big and 
upgrade to business class (it is the trip of a lifetime)! Pay extra to ship your equipment, hire a 
Sherpa and a guide service (at least $40,000), pay the $10,000 fee to even take a step into the base 
camp area, and take three weeks off of work — the longest vacation time you've spent away. 

(U) Everyone knows you've been training for this and a daily Facebook countdown has spread the 
word to all of your friends and family who await to see a picture of you at the top of Mount 
Everest. Years of hard work, careful planning, and tens of thousands of dollars have gone into this 
trip -- there's no turning back. You arrive at the mountain, settling in a base camp and it's better 
than you can imagine. You are about to realize the dream. Then, die weadier starts to turn. 

(U) First it's some rain. Then the blue sky turns gray. The Sherpa looks nervous. The wind picks up, 
snow starts, and when asked if anyone wants to turn back, you reply, "No way! The weather will 
pass! Ill never get this opportunity again and I can't turn back now." You and the group push 
ahead tiirough everydiing as die weather gets worse. 

(U) The book "Into Thin Air" describes a real-life example. In 1996, "...of die twenty climbers who 
fought dieir way to the summit (during a rogue storm), five died. One of the survivors had such 
severe frostbite, that his right forearm, nose, and most of his left hand had to be amputated." They 
continued to the summit despite the clear and present dangers, at significant cost. Imagine being a 
member of diis trip, so dedicated to reaching die summit diat you condnued ahead even as 
one fourdi of the dimbing team died. Continuing to reach the goal resulted in catastrophic 
failure, despite achieving the goal. 

(U) Mountaineers call diis phenomenon "summit fever" -- when an "individual becomes so 
fixated on readiing die summit diat all else fades from consciousness . 11 * I think part of this 
phenomenon is due to die high level of investment (monetary and spiritual) in the project that 
pushes people to make decisions that are not otherwise supported by objective data. 

(C//REL) I believe that SIGINTers, like the world-class climbers, are not immune to summit fever. 
It's easy enough to lose sight of the bad weadier and push on relentlessly, especially after pouring 
lots of money, time, and resources into something. From turning off a database or collection site to 
starting over from scratch on a target set or software code, it's difficult to let go of the dream and 
your work so far. Sometimes, however, it's the very best decision to make. Just like in summit 
fever, failing to reach a goal that no longer makes sense to pursue takes some perspective and one 
must "be okay" with failing. 

(U) So, when the summit is the only thing you can see, it's time to step back and check the weadier. 
Find those experienced "sherpas" in the organization, and ask for their honest opinion and give 
them permission to speak freely. We should not fear failure, because often failure teaches us 
something very important. Instead we should focus on "failing quickly" and "less expensively" to 
progress faster and fardier. 



* (U) Perkins, D.N., Holtman, M.P., & Murphy, J.B. (2012), " Leading at The Edge: Leadership 
Lessons from the Extraordinary Saga of Shacklet on's Antarctic Expedition. " AMACOM, p. 70. 

(U) Thoughts on this topic? Post them on the related Tapioca Pebble . 



(U) The SIGINT Philosopher: Too Many Choices 



FROM: (U//FOUO)| 
Run Date: 01/18/2011 



(U//FOUO) Editor's note: SID today soimhtQut a^SKMNT philosopher 11 in our midst, and we found 
him! We are pleased to announce that|^^^^^^^| (pictured), a Russian language analyst, will 
be penning a monthly column on that theme, starting today. The below column is unclassified in its 
entirety: 



I'm not a pleasant person to shop with. 

The reason is simple — an overabundance of choices. The second I’m placed in front of a plethora 
of tastes, textures, prices, and sizes, I lose my bearings. I don't know the difference between 
preserves, jams, jellies, or spreads. I know what I like on toast, but heaven help me if I have to go 
find it in the aisle. Do I go with the bigger jar? What about sugar-free? Swirled with peanut butter? 
Am I safe with a store brand, or can I assume that with a name like 11 S mucker's, 11 it has to be good? 

You may be wondering right about now just what any of this has to do with SIGINT... 

Plenty. We in the agency are at risk of a similar, collective paralysis in the face of a dizzying array 
of choices every single day. 

"Analysis paralysis" isn't only a cute rhyme. It's the term for what happens when you spend so 
much time analyzing a situation that you ultimately stymie any outcome. It's what happens inside 
your grandfather's brain while you wait endlessly for him to make his move on the chess board. It's 
what happens when I stand in front of the jams and jellies at the supermarket. And it's what 
happens in SIGINT when we have access to endless possibilities, but we struggle to prioritize, 
narrow, and exploit the best ones. 

But hey, we're only human, right? That's my point exactly. Ever wonder why the Pepsi Challenge 
only had two colas to pick from, or why taste-tests don't ask you to sample dozens of flavors? It's 
because we're cognitively wired to consume only so much information when making crucial 
decisions. It's why the best tasting sample will win by leaps and bounds against two competitors, 
but only by a small margin over many. 

In 1971, Herbert Simon could not have known how prophetic the following statement was, given 
that he was still two decades away from the dawn of the information age: 

"...fn an information-rich world, the wealth of information means a dearth of something else: a 
scarcity of whatever it is that information consumes . What information consumes is rather 
obvious: it consumes the attention of its recipients. Hence a wealth of information creates a 
poverty of attention and a need to allocate that attention efficiendy among the overabundance of 
information sources that might consume it ” 

(Designing Organizations for an Information-Rich World , 1971) 

In an evolving climate where there are few constants, heuristics (rules of thumb) are of little help in 
drilling down to our most useful accesses and fulfilling our mission. We spend every day trying to 





solve what sociologists call "wicked problems." They're tagged as such due to the changing nature 
of requirements, capabilities, and priorities. They have no ultimate solution and no way to 
concretely verify the quality of the outcome. These are problems with many "best" answers, and a 
far greater number of "good" answers. When tackling such issues, narrowing the playing field can 
be equally vital to broadening our expertise. 

The SIGINT mission is far too pressing for many team-building activities or brain-storming 
sessions aiming to improve our organizational approach to analysis. At the same time, the SIGINT 
mission is far too vital to unnecessarily expand die haystacks while we search for the needles. 
Prioritization is key. 

And now, if you'll excuse me, I'm off to the store for some jam... 




(S//SI) Dealing With a ’Tsunami 1 of Intercept 



FROM : 

Human Language Technology (S23) 
Run Date: 08/29/2006 



(S//SI) Everyone knows that analysts have been drowning in a tsunami of intercept whose volume, 
velocity and variety can be overwhelming. But the Human Language Technology Program 
Management Office (HLT PMO) can predict that in the very near future the speed and volume of 
SIGINT will increase even more, almost beyond imagination. And we are working on ways to 
help analysts deal with it all. 

(S//SI) Of the HLT PMO's five Strategic Thrusts, the one that addresses this problem is High 
Speed/ High Volume. It must deal with today's collection and must plan for tomorrow's. The 
current collection environment is characterized by huge amounts of data, coupled with severely 
limited capability to send material forward, and extremely limited number of queries that exactly 
describe messages of value. That means we are capable of finding huge amounts of data, much of 
which is not what we really want, and that we cannot send it all back for analyst processing. 

(TS//SI) To plan for tomorrow, High Speed/ High Volume is in line with changes in the overall 
NS A/CSS systems, particularly TURBULENCE and FURMOIL because when they become a 
reality in the near future, we can expect collection capabilities to increase significantly, 
TURBULENCE is an umbrella cover term describing the next generation mission environment that 
will create a unified system. TURMOIL is a passive filtering and collection effort on high-speed 
networks. This is designed to be flexible and can be modified quickly to deliver data in analyst- 
ready form. 

(S//SI) One of High Speed/ High Volume's first efforts is in developing and implementing ways to 
push HLT capabilities very dose to the collection points of the SIGINT system. In particular, 
HLT is about to demonstrate an operational prototype of language identification for Special Source 
Operations (SSO) Counterterrorism text targets running at line speeds (STM- 16) at the packet- 
level. Resources permitting, HLT analytic processors will automatically generate content-based 
events for TURMOIL based on language. 

(S//SI) HLT processors will demonstrate the ability to characterize very high speed channels based 
on content, thus enabling analysts to task the SIGINT system to send back messages based on 
information found in message content, not just on externals. (Externals can be Signal Related 
Information (SRI) that comes with each message, such as channel, Time Up/Time Down, etc.) 
Using HLT services, analysts will be able to build more precise descriptions of the data they want. 
In addition, content-based metadata will allow SIGDEV analysts to run more detailed surveys. HLT 
services that work on data content at the collection point can also provide indications or warnings 
that the SIGINT system must adapt its collection strategy. 

(S//SLVREL) Resources permitting, High Speed/ High Volume will deploy capabilities for voice, 
text, and image data, and will take advantage of research being done by a number of organizations 
including the Research Directorate's Coping With Information Overload Office (R6), Disruptive 
Technologies Office (DTO), and SID/ Analysis and Production's Advanced Analysis Laboratory 
(AAL). HLT research and transfer of its technology into operations means the development of 
algorithms that can incorporate HLT capabilities for the processing of elements such as email 
attachments and VOIP. 




(S//SI/REL) The research and technology transfer also may provide "stealthy," low-profile in-target 
implants for Tailored Access Operations (TAO) or technologies to enable high speed processing in 
very low size, weight and power applications for other CLANS IG customers. And, to help address 
the "unknown unknown" target analysis problem, HLT is investigating techniques and technologies 
for high volume voice processing so that all voice data can be scanned for key words before it is 
selected based on phone numbers. 

(S//SI) Ultimately, H LTs High Speed/ High Volume will give the analyst greater ability to 
influence collection and processing much farther forward in the SIGINT system, as well as help the 
SIGINT system achieve greater overall filtering and selection effectiveness. That means more 
analysts wil be getting better SIGINT at a time when volume and velocity are maximum. 




